test: fix flaky request ordering tests in shared RQ access mode#931
Merged
Conversation
In shared request queue access mode, the relative order of two forefront operations performed close together is not guaranteed due to the API propagation delay, so the test now only verifies that both forefront requests come before the regular one. The strict ordering assertion is kept for the single access mode, which is deterministic.
vdusek
added
adhoc
github-actions
Bot
added
the
tested
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #931 +/- ##
==========================================
- Coverage 86.94% 86.87% -0.07%
==========================================
Files 48 48
Lines 2942 2942
==========================================
- Hits 2558 2556 -2
- Misses 384 386 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Apply the same shared access mode relaxation to the remaining order-sensitive tests: the relative order of requests added close together (forefront or regular) is not guaranteed due to the API propagation delay, so assert membership instead of exact positions in shared mode. The strict ordering assertions are kept for the single access mode.
- Wait for forefront operations to propagate to the queue head in shared mode before draining the queue, so the forefront-before-regular boundary assertions are robust against the same propagation delay (issue #808) that reorders forefront operations relative to each other. - Add the same propagation wait to test_request_reclaim_with_forefront, which had the identical reclaim-to-forefront-then-fetch strict ordering assertion and was missed by the original relaxation. - Assert len(fetched_urls) == 5 in test_forefront_requests_ordering so the set-based shared-mode assertions cannot mask duplicate or extra fetches. - Deduplicate the propagation-delay rationale: extend the module-level note and point the inline comments at it.
vdusek
changed the title
vdusek
added a commit
that referenced
this pull request
Jun 5, 2026
…flake via polling (#920) ## Summary The SDK counterpart of apify/apify-client-python#844: introduce a single shared `tests/_utils.py` with one polling helper, and migrate the whole test suite to it — replacing fixed sleeps and hand-rolled retry loops that caused flakiness. ## Changes - **Shared `tests/_utils.py`.** One `poll_until_condition(fn, condition=bool, *, timeout, poll_interval, backoff_factor)` helper — identical to the one in apify-client-python#844 — polls a sync-or-async callable until a condition holds or a wall-clock timeout expires. `backoff_factor=2` subsumes the former `call_with_exp_backoff`; `timeout=0` is the "call once" case. The `maybe_await` adapter, `generate_unique_resource_name`, and the shared RSA crypto test keys move here too. The per-package `tests/integration/_utils.py` and `tests/e2e/_utils.py` are removed, and cross-test imports (e.g. the crypto keys, previously imported from the `test_crypto` module) now go through this single module. - **Request queue tests.** ~50 polling call sites in `test_request_queue.py` migrated to `poll_until_condition`. The single/shared timeout is centralized in an `rq_poll_timeout` fixture (`timeout=0` single, `timeout=30` shared), backed by an `rq_access_mode` fixture — replacing the access-mode derivation block previously copy-pasted into every test. The shared-mode request-ordering relaxations from #931 are preserved on top. - **Deflaked tests.** Fixed the flaky `test_actor_adds_webhook_and_receives_event` e2e test (the client now stays alive 5 s after `add_webhook`, and the unbounded `INITIALIZED` startup loop becomes bounded polling) — replaces #930. Replaced the `retry_counter` loops in `test_actor_charge.py` (4×) and fixed sleeps in `test_actor_lifecycle`, `test_actor_key_value_store`, and `test_apify_event_manager` with condition polling. Fixed sleeps that are semantically required are intentionally kept: negative checks ("event must NOT fire"), mtime-granularity checks, simulated latency, and sleeps inside deployed Actor `main` bodies (where the helper is unavailable). Verified: lint, type-check, and the full unit suite pass; integration/e2e require a platform token and run in CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
test_request_ordering_with_mixed_operations[shared]flaked on master (CI run) with the reclaimed forefront request being fetched before the newly added forefront request.test_request_ordering_with_mixed_operations,test_forefront_requests_ordering) now only verify that forefront requests come before regular ones in shared mode, asserting membership instead of exact positions. The strict ordering assertions are kept for the deterministic single access mode.test_request_reclaim_with_forefronthad the identical reclaim-to-forefront-then-fetch strict ordering assertion and gets the same propagation wait (its assertion stays strict, since after propagation it is deterministic).test_forefront_requests_orderingnow assertslen(fetched_urls) == 5, so the set-based shared-mode assertions cannot mask duplicate or extra fetches.[shared]variant passed against the real API (before the follow-up commit, which only makes the assertions stricter/more robust).